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What is good literature? Are the characteristics that make literature 
good or poor in one genre the same characteristics that affect quality 
ixL another genre? These are questions that, when answered, will greatly 
aid teachers, for once it is known what makes the "good" poem, the "good" 
short story, the "good" junior novel, then teachers will have some clear 
guidelines concerning what to teach when dealing with these types of 
literature. Teachers will then know that the poem is a good vehicle with 
which to exemplify literary characteristics X and Y; the junior novel is 
a good vehicle for teaching characteristics Y and Z, and so on. 

Unfortunately there has been no empirical research aimed at identifying 
significant literary traits across genre or for any one type of literature. 
This study was an attempt to take the first empirical step in that 
direction. The researchers attempted to answer the basic question, "What 
are the literary characteristics which constitute 'good literature' within 
the genre of the junior novel?" 
Literary Characteristics 

A review of current opinion on the judging of literary quality 
revealed that the number of traits associated with good literature has 
almost a one to one relationship with the number of people who have 
opinions. Probably the most frequently mentioned attributes are: 

1. THEME 

2 . CHARACTERISTICS 

3. PLOT 

4. STRUCTURE 

5. STYLE 

For operational definitions of the above mentioned traits, the researchers 
used the descriptions given by Dunning and Howes (1975, pp. 205-206). 
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A review of high school literature texts revealed two more variables 
that are traditionally associated with quality of literature: 

6. SETTING 

7. LITERARY DEVICES 

The seven above mentioned characteristics seemed to adequately 
represent the more general variables associated with quality of literatxire 
for novels. The researchers also attempted to include variables that 
pertained to particular syntactic and semantic aspects of an author's 
writing. To represent those aspects tht; researchers chose the following 
variables: 

8 . READABILITY 

9. WITHIN SENTENCE MODIFICATION (WSM) 
10. BETWEEN SENTENCE MODIFICATION (BSM) 

Readability was defined as the difficulty of a book as measured by the 
Fry (1968) formula. WITHIN SENTENCE MODIFICATION is the extent and depth 
to which clauses y phrases and single words are used as modifiers within 
a sentence. BETWEEN SENTENCE MODIFICATIONS is the extent and depth to 
which sentences within a paragraph act as modifiers of the topic sentence. 
WSM and BSM have been shown to be significant predictors of .composition 
quality (DiStefano and Marzano, 1976) and of paragraph readability (Marzano, 
1976). These measures have also been postulated by Christensen (1963) 
to quantify elements which are basic to quality in professional writing. 
The Sample and Procedure 

Ten books from the domain of the junior novel were selected for 
analysis. They were: 

Man Without A Face 
Soimder 
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Shadow of Bull 

Don^t Play Dead Until You Have To 

Mrs, Frtsby and • the Rats of Nlmh 

1*11 Get There but it Better Be Worth the Trip 

Pigman 

Enchantress From the Stars 

Soul Brother and Sister Lou 

That Was Then , This Is Now 
Eleven raters Independently rated the books on OVERALL QUALITY using 
a Likert type rating. The raters then calculated measures 8-10 for each 
book and rated each book on measures 1-7 (Likert Scale). The raters 
waited two weeks between the time they assigned quality ratings and the 
time they assigned ratings for measures 1-7. This was to .assure some 
independence between OVERALL QUALITY and the other measures. Prior to 
assigning values for measures 1-7 the raters were trained as to the 
operational definitions of the qualities being measured and the guidelines 
for assigning values for each measure. For example, for THEME the raters 
were instructed to answer the following questions as a guide to assigning 
weights on the Likert scale. 

a. Is the theme clear? Does it dominate the book 
and seem didactic? 

b. Is the theme important — appropriate to young people, 
to society's values, worth thinking about? 

c. Does the theme offer the reader an honest perspective 
on the values of human experience? 

To insure Independence among measures 1-7 all books were rated on one 
measure before the next measure was calculated. After all ratings were 
made the average reliability for each rater (Winer, 1971, pp. 283-296) 



was calculated on each measure along with the reliability of the combiaed 
ratings for each measure (eleven ratings combined for each measure). 
Those reliabilities are reported in Table 1. 



Table 1 

Average and Combined Ratings For Each Measure 





Average Reliabilities 


Combined Reliability 


THEME 


.40 


.83 


CHARACTERIZATION 


.44 


.89 


PLOT 


.55 


.91 


STRUCTURE 


.42 


.85 


STYLE 


.58 


.92 


SETTING 


.59 


.92 


LIT DEVICES 


.59 


.94 


READABILITY 


.58 


.94 


WSM 


.58 ^ 


.93 


BSM 


.60 


.95 


OVERALL QUALITY 


.61 


.96 



As is evidenced by Table 1, when all eleven ratings, for any given 
book on any given measure, are considered as a composite rating the 
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reliability of that rating is very high — usually above .90. This is a 

statistical phenomenon. The reliability of a composite rating will 

increase as one adds raters. The average rating for individual raters is 

probably the most interesting aspect of Table 1. Considering the unreliabibility 

generally associated with Individual ratings (Diederich, 1961; Follman and 

Anderson, 1967), the average ratings reported for this study are quite 

high. They are high enough, in fact, that the researchers decided to 

consider each rating as an independent observation rather than combine 

ratings for a given measure and a given book. This is tantamount to having 

eleven independent observations for each book on each measure. Given the 

ten bookS; the researchers were able to analyze each measure on the basis 

of 110 observations. This greatly increased the scope of statistical 

analyses thar> could be performed. 

Analysis of the Data 

The data was analyzed in three ways: 

1. one way ANOVAs were conducted using the books as 

the independent variable and measures 1-10 and QUALITY 
as the dependent "variables 

2. bivariate and multiple regression analyses were 
performed using measures' 1-10 as predictors and 
OVERALL QUALITY as the criterion. 

3. a principal factor — factor analysis (Harman, 1967) 
with a varimax rotation was conducted using OVERALL 
QUALITY and measures 1-10. 

The ANOVA was used to calculate the average and composite reliabilities 
for the measures (Winer, 1971) and to answer the following research 
question: 



1. Is there a significant (.05) difference among the books 
on OVERALL QUALITY and the 10 measures? 

The factor analysis was used to answer the question: 

2. What do the ten Indices measure In common with one 
another and with OVERALL QUALITY? 

The regression analyses were used to answer the questions: 

3. Which measures have the strongest relationship with 
OVERALL QUALIIT:? 

4. What Is the extent of Interrelationships among the 
ten measures? 

Results and Discussion 

The ANOVAs using QUALITY and the ten Indices as dependent measures 
showed significant differences among means for all dependent measures. 
Thus the answer to the first research question was affirmative™ there Is 
a difference on all measures among the ten books. Since the ANOVA process 
tests the null hypotheses that the means for the books are equal, the 
reliabilities for the measures used to answer research question 1 are 
those for the combined ratings (see Table 1) . Those reliabilities are all 
very high. Thus, the researchers concluded that the ten books were a 
random sample,^ from the domain of the junior novel, that was representative 
of truly different levels for the literary characteristics measured by the 

i 

ten indices and OVERALL QUALITY. This, of course, greatly, increased the 
external validity of the findings on research questions 2-4. 



Because the books were a sample from the domain of the junior novel, a 
random effects model was used for the ANOVAs. 
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The principal factor — factor analysis with varimax rotation 
(question 2) produced two very clear factors. The loadings for the 
rotated factors are reported in Table 2. 



Measure 


Table 2 
Varimax Rotated Factors 
I 


II 


OVERALL QUALITY 


.92 


-.16 


READABILITY 


-.17 


.76 


WSM 


-.42 


.45 


SETTING 


.77 


.02 


PLOT 


.79 


-.27 


CHARACTERIZATION 


.59 


-.29 


THEME 


.69 


-.08 


LIT DEV 


.82 


-.06 


STRUCTURE 


.79 


-.19 


STYLE 


.89 


.01 


BSM 


.07 


.73 


Eigenvalues 


5.51 


1.30 


% of Variance 


50.1% 


11.8% 



y 

Because each rating was considered an observation, a w incipal factor rather 
than principal component factor analysis was used to c":ntrol for the 
unreliability of the individual ratings. 
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OVERALL QUALITY, SETTING, PLOT, CHARACTERIZATION, THEME, LIT DEV, 
STRUCTURE and STYLE all loaded quite heavily (higher than .50) on the 
first factor (column I), REAT ABILITY, WSM and BSM loaded heavily on the 
second factor. This indicates that WSM, BSM and READABILITY all measure 
one characteristic in common and the other indices measure something else 
in common — something that is different from what is measured by WSM, 
BSM and READABILITY. The researchers concluded that within/between sentence 
modification and readability have little to do with the quality of a/ 
junior novel. This is not to say that authors of the junior novel do not 
use a great deal of within and between sentence modification. Indeed, 
the mean for WSM was 2.7 and for BSM 3.1. These averages are very high 
when compared with the writing of high schoolers and adults across the 
nation (Marzano and DiStefano, 1976) . The fact that WSM and BSM do not 
measure the same trait (for junior novel^) as OVERALL QUALITY but have 
higji means, indicates that authors of the junior novel are all adept at 
using within and between sentence modification but the extent to which 
those skilled writers use modification does not enhance or detract from 
the quality of the novel. The mean for READABILITY was 6.7. Considering 
that the books were written for a junior higih school audience, this indicates 
that the authors were able to write high quality stories and still keep 
the language and syntax at an understandable level. 

The bivariate regression analyses (questions 3 and 4) using OVERALL 
QUALITY as the criterion and the ten measures as predictors produced the 
results summarized in Table 3. 
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Table 3 

Bivariate Regression: QUALITY as Criterion 



Predictor 


beta-weii?ht 


probability 


READABILITY 


-.26 


.009 


WSM 


-.43 


.001 


SETTING 


.71 


.000 


PLOT 


.78 


.000 


CHAR 


.55 


.000 


THEME 


.63 


.000 


LIT DEV 


.74 


.000 


STRUCTURE 


.75 


.000 


STYLE 


.81 


.000 


BSM 


-.08 


.402 



Since beta-weights in a bivariate regression analysis are equal to 
the correlations between predictor and predicted, the second column of 
Table 3 can be used to answer research question 3, which measures have 
the strongest relationship with OVERALL QUALITY? In rank order of their 
strength of positive relationship with quality, the measures are: 

STYLE ,81 

PLOT ,78 

STRUCTURE ,75 

LIT DEV .74 

SETTINQ ,71 

THEME ,63 

CHAR .55 



BSM -.08 
READ -.26 
WSM -.43 

This can be interpreted as a hierarchy of literary traits which work 
together to constitute good quality in the junior novel. 

A multiple correlation was calculated among QUALITY and the 
ten measures. That multiple R was .92. This means that the ten 
measures account for 84% (R^) of the variance in OVERALL QUALITY. Roughly 
speaking this indicates that the ten measures used in the study account 
for 84% of whatever determines quality in the junior novel. 

The multiple regression analysis also produced information useful 
in answering research question 4 concerning the Interrelationship among 
the ten measures. When entered into a multiple regression equation with 
OVERALL QUALITY as the criterion, only five out of the ten measures were 
significant predictors of quality. Those measures were STYLE. PLOT. SETTING. 
THEME and STRUCTURE. A logical question might be. how can a measure be a 
significant predictor ofQUALITY in a bivariate regression analysis yet not 
be a significant predictor in a multiple regression analysis? The 
explanation rests in the interrelationship or collinearity among predictors. 
CHARACTERIZATION was not a significant predictor in the multiple regression 
analysis because it was highly correlated (.53) with STYLE. Hence, 
when STYLE was entered into the equation It "used up'/ so to speak, most 
of the predictive power that CHARACTERIZATION possessed. LITERARY 
DEVICES was not a significant predictor in the multiple regression 
equation because it had high correlation with SETTING. PLOT. STRUCTURE and 
STYLE (.57. .64. .64. .73 respectively). 

READABILITY. WSM. and BSM were highly intercorrelated and also had weaker 
relationships with the criterion. 
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The results of the multiple regression analysis somewhat change 
the Interpretation of the hierarchy of literary qualities mentioned 
previously. Since CHARACTERIZATION and LITERARY DEVICES measure the same 
traits as other measures, they should probably be dropped from the 
hierarchy, READABILITY, WSM and BSM also measure a common trait — a trait 
which is not related to quality at the junior novel level. Hence, the 
hierarchy of literary characteristics in their order of importance is: 

STYLE 
PLOT 

STRUCTURE 

SETTING 

THEME 

When only these measures are entered in a multiple regression 
equation with QUALITY, the multiple R is .86. 
Implications 

The results of the study were surprisingly clear cut and interpretable. 
Quality in the junior novel is basically a function of the author's 
style, plot, structure, setting and theme. For the teacher this means that 
when dealing with the junior novel the above mentioned literary characteristics 
are those that should be stressed because good vs. poor junior novels are 
differentiated in the extent to which style, plot, structure, setting and 
theme are developed. The negative counterpart of this is that the junior 
novel is a poor vehicle with which to teach literary devices, characterization 
and the different ways a writer can use modification within and between 
sentences. These are not distinguishing factors within the realm of the 
junior novel. 

Thus, this study has focused the types of Instructional goals a 



teacher should have when dealing with the junior novel. If all types of 
literature were analyzed in a fashion analogous to this study, teachers 
might someday possess clear guidelines regarding which type of literature 
to use when teaching each type of literary characteristic. This study 
has provided a straightforward answer for the junior novel. Perhaps 
future studies will provide the answers for other genre. 
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